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ABSTRACT 


Recent experience at ESD in acquiring complex computer based systems 
has identified a deficiency in existing systems management techniques 
in the area of computer programs. The systems management techniques 
generally in use were designed for "equipment 11 systems and need to be 
expanded to include computer programs. This paper describes an ESD 
approach to adapting existing AFSC system management technioues to 
computer programs. Procedures for insuring system compatibility, 
design integrity and technical control are discussed and a method 
for achieving design verification and qualification is presented. 
Particular emphasis is placed on the relationship of these techniques 
to computer programs as elements of large computer based systems. The 
application of these techniques is illustrated through selected examples 
taken from current ESD system procurements. 
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SUMMARY 


Recent experience at ESD in acquiring complex computer based systems nas 
identified a deficiency in existing systems management techniques in the 
area of computer programs. The systems management techniques generally 
in use were designed for Equipment 11 systems and needed to be expanded 
to include computer programs. This paper describes an ESD approach to 
adapting existing AFSC systems management techniques to computer programs. 
Procedures for insuring system compatibility, design integrity and 
technical control are discussed and a method for achieving design 
verification and qualification is presented. Particular emphasis is 
placed on the relationship of these techniques to computer programs as 
elements of large computer based systems. The application of these 
techniques is illustrated through selected examples taken from current 
ESD system procurements. 


INTRODUCTION 


Systems Management 

The concepts of "Systems Management” within the Air Force are well 
established in the 375-series Air Force regulations and Systems Command 
manuals. After many years of experience the procedures for managing 
systems acquisition have become highly structured and highly detailed 
in some areas. At the same time the systems management structure has 
been kept general enough so that the techniques can be easily applied to 
any large system (whether it be a missile system, an electronic system, 
an aeronautical system, etc.) and can be selectively applied to small 
procurements. 

Effects of Computer Programs 

Since the development of the Semi-Automatic-Ground-Environment (SAGE) 
system in the late 50*s, the use of digital computers and computer programs 
has played an increasing role in military systems. Here at Electronic 
Systems Division (ESD) almost every "L" system developed in recent years 
has been intimately involved with computers and computer programs. In 
spite of the extensive application of computer programs, their role in 
systems has not been well defined nor generally understood. Typically, 
"Systems Management" has been applied to the collection of equipment, 
facilities, personnel, doc lamentation, etc., that comprise a system, without 
regard for the computer programs within the system. To some extent this 
may be due to the self-imposed independence of computer programmers and 
analysts from the engineering discipline. But to a greater extent it is 
due to a lack of understanding of computer programs and their design and 
application. In part, this lack of understanding is because of the 






inherent properties of a computer program. The computer program is an 
elusive and intangible object. It cannot be readily seen or felt and 
thus is difficult to describe. 

This lack of understanding has not, however, prevented computer 
programs from becoming an important element of many current systems. 
Unfortunately, the Systems Management techniques have not yet recognized 
the impact of computer programs on systems development. Thus computer 
programs in the past have failed to receive the proper "systems 11 emphasis 
required to effectively utilize them within a system. Typically, a system 
under development has progressed well into the detailed design stage and 
often into fabrication while the computer programmers have been left in a 
vacuum to design and code the computer program with only a minimum of 
guidance. Mien the equipment and computer programs are integrated, the 
problems start: Functions that each group (engineers and programmers) 
thought the other was responsible for often are not being performed at 
all* interfaces between computer programs, equipment and personnel are 
incompatible• essentially, the system will not work. The result is often 
extensive redesign of equipment and computer programs with an accompanying 
increase in cost and delay in schedule. Frequently all or most of the 
redesign effort is placed on the computer programs because of their 
inherent flexibility. Continued capitalization on the flexibility of 
computer programs to correct system deficiencies, without due consider¬ 
ation of systems effectiveness, will eventually place severe limitations 
on the computer programs within the system. 

In the past year the Technical Requirements and Standards Office of 
ESD has undertaken the task of expanding the established systems management 
techniques to include computer programs as an essential element of the 
system. The result of this effort has taken the form of supplements^* * 
to the AFSC 375 series manuals that can be used as the basis for future 
revisions to these manuals. Essentially the activities of systems 
management have been applied to the fundamental elements of systems as 
illustrated in figure 1. Typically, an Air Force System Program Office 
is organized along the five sub-areas of management shown in figure 1. 

Activities represented under Program Control and Procurement and 
Production tend to be of an administrative nature, covering such matters 
as budget, schedules, costs and contracting. 

As depicted, the major task is System Engineering. This activity 
accomplishes the primary job of maintaining technical control of the 
system program in all its aspects. 

Configuration Management and Test and Deployment are closely 
associated with System Engineering, but are separated for special 
consideration and responsibility. They pertain, most directly, to the 
major system elements: equipment,facilities, and computer programs. 
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The inclusion of computer programs as a major element of the system, 
as illustrated in figure 1, represents a point of departure from the 
existing systems management techniques. 


COMPUTER PROGRAMS WITHIN A SYSTEM 


What is a Computer Program 

Definitions of computer programs are as varied as the systems within 
which they function. Within the context of systems management, however, 
we do not require a concise technically accurate definition that is 
acceptable to all, but we must limit computer programs to some class of 
objects that we can effectively manage. For this purpose, computer programs 
are defined as a sequence of coded instructions and data contained on 
magnetic tape, punched cards or some other appropriate medium in a form 
suitable for insertion into a digital computer. Within the contractual 
entities defined by the Air Force, i.e. manufactured products, data 
(documentation), and services, the computer program possesses properties 
similar to a manufactured product and similar to data items. Because of 
the similarities to equipments and a desire for effective technical 
control, the Air Force and NASA have chosen to class all computer programs 
as manufactured products, i.e. contract end items. 

A Functional Element . The computer program is as much a functional 
element of a system as is a facility or a piece of equipment. The computer 
program usually performs functions that were performed by equipment or 
personnel in the past. It generally performs these functions with more 
speed or accuracy than was previously available and thus has become an 
essential element of the system. In many instances the computer program 
is basically a set of automatic operating procedures. 

System Compatibility . Since the computer program is a functional 
element of the system, it must interface with other system elements. As 
with other system interfaces, all of the computer program interfaces must 
be accurately defined throughout the system design process. The obvious 
interface between the computer program and the computer is only the 
beginning, for the computer program will also interface with other 
computer programs, external equipment and personnel. 

Design Integrity . Throughout the design and development of computer 
programs, emphasis must be placed on design integrity. Is the computer 
program, as designed, cost effective in terms of system timing, use of 
available computer memory, etc.? W T ill the computer program satisfy all 
of the design requirements? These and other questions must be continuously 
asked throughout the design and development process. 







Performance Verification . The complexity of a computer program 
requires that a systematic test program be used to determine compliance 
with contractual requirements. The performance of both individual 
computer programs and the total system must be verified. It is not 
unusual for the testing phase of computer program design and development 
to represent 50 % or more of total computer program costs. 

System Implications 

The definition of computer programs as deliverable contract end items 
is the basis for including computer programs in the systems approach. 
Established systems management techniques must be tailored, though, to 
effectively manage computer programs within a system. The systems 
approach must be tailored to take advantage of the similarities between 
equipment and computer programs while catering to the uniqueness of 
computer programs. 

Support Items . Computer programs require most of the support items 
that are normally required for equipment. Thus operator handbooks, 
manuals, etc. must be written and verified for computer programs. 

Training requirements must also be considered as well as manning require¬ 
ments for the operational system. In addition, expendable supplies such 
as magnetic tape, punch cards, etc. must be made available. 

Production . Production, in the sense of manufacturing a quantity of 
"Chinese copies 11 of a piece of equipment, does not apply to computer 
programs. Once the initial design and development is concluded and the 
computer program is qualified, production is completed. Reproduction of 
a computer program on magnetic tape or a deck of cards to obtain an 
identical copy is a relatively simple and inexpensive process involving 
only peripheral computer equipment. 

Spare Parts . Unlike equipment, computer programs do not wear out 
or degenerate as a function of time. Unless tampered with, a sequence of 
computer instructions will continue to perform the same function endlessly 
until the computer program is affected by some external source. True, 
failure may occur within a computer program, but the failure is always 
due to a latent design deficiency. Obviously, then, spare parts are not 
required and provisioning, useful life, interchangeability, etc. do not 
apply. Since the computer program does not require spare parts, 
maintenance in the accepted sense of the word does not apply. There is 
however, a term Maintenance of computer programs” that refers to the 
continual process of correcting latent deficiencies and implementing 
modifications within computer programs. 

Reliability . Since a computer program never wears out it is virtually 
impossible to predict or analyze failure rates. Any failure of the computer 
program is a latent design deficiency and its occurrence cannot be predicted. 
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It is obvious then that a computer program cannot be designed for reliability 
and cannot be tested or evaluated for reliability. Reliability does not 
apply to computer programs as end items although the computer programs 
may be used to enhance system reliability. 

Impact . The definition of computer programs as contract end items 
within a system provides a vehicle for emphasizing this important system 
element. Sufficient analysis conducted early in the system design can 
identify those system requirements that will affect the computer program 
design. If, for example, modular computer programs are required or 
multiprogramming is needed, these requirements can be identified before 
the detailed design of computer programs commences. This approach allows 
the Air Force to satisfy many of the objectives of DOD directive 3200.9-^ 
such as establishing firm and realistic performance specifications, 
precisely defining interfaces and responsibilities, identification of 
high risk areas, and establishing firm and realistic schedules and cost 
estimates during the acquisition of computer programs. 


THE SYSTEMS APPROACH 


Specification of Requirements 

In applying the systems approach to computer based systems the 
computer programs must be given "equal time" even as early as the 
conceptual planning for the system. Early conceptual studies must 
consider the computer programs as a vital element of the system. The 
effects of performing functions by automated methods, as well as electrical 
or manual methods, must be considered. The system specification should 
identify all of the system/design requirements for the total system. In 
allocating the system requirements to system segments and contract end 
items, extensive analysis of the trade-offs between equipment, computer- 
programs and personnel must be conducted. The result of this system 
analysis effort is to establish a system specification that identifies the 
system performance/design requirements, identifies all of the system segments 
and contract end items and allocates the design requirements to these contract 
end items. The system specification represents one of the first important 
steps in the system development process. As shown in figure 2 the system 
specification is the basis for all of the system design and development 
effort that follows. 

From the system specification the individual contract end item 
specifications are developed. It is this preparation of a design 
specification, containing performance/design and test requirements, that 
is the key to applying systems management to computer programs. The 
design-to specification, or part I computer program contract end item 
(CPCEI) specification, 1 contains all of the performance, design and test 
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requirements for an individual computer program contract end item. The 
specification must identify and define all of the interfaces between the 
computer program CEI and other computer program and equipment CEI's. The 
design specification, once approved^ will control the development of that 
computer program. Thus, the computer program CEI will be designed and 
qualified against its individual design specification. 

Design Integrity 


Throughout the development process the customer (in this case the 
government) should actively monitor the contractor's efforts in developing 
the system. If the contractor is left to work in a vacuum, as has 
happened in the past, the delivered system often does not satisfy the user. 
Both the contractor and the customer can benefit from the feedback 
provided by an exchange of technical information throughout the design 
and development process. In recent years this exchange of information on 
military systems has been provided by the conduct of technical design 
reviews- 1 - at the predetermined times in the process. The application of 
these design reviews to computer programs^ provides the technical manager 
with a tool to assist in establishing the design integrity of computer 
programs within a system. The relationship of these design reviews to 
the system development process is indicated in figure 2. 

System Design Review . The purpose of this first review is to study the 
contractor 1 s system design approach. At the SDR a critical examination of 
the system design is performed to insure that a proper understanding of all 
design requirements exists. An analysis of contractor documentation in the 
form of functional diagrams, trade-off study reports, schematic diagrams, 
initial design specifications, etc. is conducted. A prime objective of the 
SDR is to review the allocation of functional requirements to the various 
system segments and contract end items. Thus, for computer programs, the 
SDR must insure that only those system design requirements that can be 
realistically satisfied by computer programs have been allocated to 
computer program contract end items (i.e. operational, utility, diagnostic, 
etc.). Prior to the conduct of the SDR, trade-off studies concerning 
equipments vs. computer programs must have been completed to provide a 
cost effective allocation of design requirements. Satisfactory completion 
of the SDR permits completion of the Part I specifications ("design to" 
specifications) for all computer program CEI's. These specifications 
form the basis for the second technical review in the design process. 

Preliminary Design Review. The Preliminary Design Review (PDR) is 
usually held within 60 days after the start of the Acquisition Phase. The 
preliminary design of the computer program CEI is in progress based on the 
approved "design to" specifications for the end item. The purpose of the 
PDR is to evaluate the design approach for the end item or group of end 
items in light of the overall system requirements} thus, the prime 
objective of the PDR is achieving design integrity. A review of the 
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interfaces affecting the computer program contract end item is an 
important element of a PDR. Emphasis is placed on verification of 
detailed interfaces with equipment and with other computer program 
CEI’s. At the PDR the instruction set of the computer to be used must 
be firmly established. The programming features of the computer, e.g. 
interrupts, multiprocessing, time sharing, etc. must be known. All 
external data formats and timing constraints must be identified. The 
computer program storage requirements and data base design are reviewed 
for technical adequacy at this time. The structure of the computer 
program contract end item is also reviewed at the PDR. During the initial 
design process for a complex end item the requirements of the Part I 
specification which are function-oriented are allocated to computer 
program components or modules. The allocation of functions to computer 
program components within the CPCEI is examined at the PDR. The primary 
product of the review at this level is establishing the integrity of the 
design approach, verifying compatibility with the Part I specification, 
and verifying the functional interfaces with other contract end items 
in order that detailed design of the computer program CEI can commence. 

Critical Design Review . The Critical Design Review (CDR) is a formal 
technical review of the design of the computer program contract end item 
at the detailed flowchart level. It is accomplished to establish the 
integrity of the computer program design prior to coding and testing. 

This does not preclude any coding required prior to the CDR to demonstrate 
design integrity, such as testing of algorithms. In the case of a complex 
computer program CEI, as the design of each component proceeds to the 
detailed flowchart level, a CDR is held for that component. In this manner, 
the CDR is performed incrementally by computer program components and the 
reviews are scheduled to optimize the efficiency of the overall CDR for the 
end item as a whole. Due to the varying complexity of the parallel design 
efforts for computer program CEI components, it would be unreasonable to 
delay all of the components being developed to hold one CDR for the computer 
program end item. 

At the CDR, the completed sections of the Part II computer program CEI 
specification (detailed technical description) are reviewed along with 
supporting analytical data, test data, etc. The compatibility of the CPCEI 
design with the requirements of the Part I specification is established at 
the CDR. f, Inter n interfaces with other CPCEI^ and "intra" interfaces 
between computer program components are examined to insure compatibility. 
Design integrity is established by review of analytical and test data, 
in the form of logic designs, algorithms, storage allocations and associated 
methodology. In general, the primary product of the CDR is to establish the 
design and development accomplished as the basis for contination of the 
computer program development cycle. Immediately following the CDR, coding 
of individual components takes place and the process of checkout and 
testing of the components begins. 
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First Article Configuration Inspection . When the design and testing 
ot the computer program CEI is essentially completed, the Part II 
Specification is available for review. The Part II specification 
provides a complete and detailed technical description of the computer 
program CEI n as built" and functions as the primary document for use by 
programmers in correcting errors and designing changes to the computer 
program CEI. The technical accuracy and completeness of the Part II 
specification must be determined prior to acceptance of the document by 
the Air Force. The First Article Configuration Inspection (FACl) provides 
the vehicle for the required review of the Part II specification and is an 
audit of the Part II specification and the computer program CEI as delivered. 
The primary product of the FACI is the formal acceptance by the Air Force 
of the Part II specification as an audited and approved document. Air 
Force acceptance of the computer program CEI for Category II testing is 
based on the successful completion of the Category I Test Program and the 
FACI, but it does not relieve the contractor from meeting the requirements 
of the system specification. Subsequent to FACI, the configuration of the 
computer program CEI is essentially controlled at the machine instruction 
level so that the exact configuration is available for Category II system 
testing. 

Design Verification 

Testing, as defined by the Air Force, is divided into^three classes 
or categories of testing, two of which. Category I and II are important 
in development testing of Air Force systems and will be discussed here. 
Category I tests for computer program CEI f s^are conducted by the contractor 
and will normally proceed in such a way that testing and functional 
demonstrations of selected functions or individual computer program 
components can begin early during acquisition and progress through 
successively higher levels of assembly to the point at which the complete 
computer program CEI is subjected to formal qualification testing. Since 
the total process is typically lengthy and represents the major expense 
of computer program acquisition for the system, the test program includes 
preliminary qualification tests at appropriate stages for formal review 
by the Air Force. While the tests are preliminary in nature (they do not 
imply acceptance or formal qualification), they do serve the necessary 
purposes of providing check points for monitoring the contractor^ 
progress tov T ards meeting design objectives and of verifying detailed 
performance characteristics which, because of sheer numbers and complexity, 
may not be feasible to verify in their entirety during formal qualification 
testing. Category II tests are complete system tests, including the 
qualified computer program end items, conducted by the Air Force with 
contractor support in as near an operational configuration as is practicable. 
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Category I (Qualification) Testing * The Category I test program 
verifies that the computer program contract end item satisfies the 
design/performance requirements of the Part I 11 design to 1 ’ specification. 

The Category I test program must be designed to insure that all of the 
functional requirements, as translated into computer program components, 
are tested and that requirements are not lost in the translation. The 
program is divided into two major classes of tests: Preliminary 
Qualification Tests (PQT) and Formal Qualification Tests. 

Preliminary Qualification Testing (PQT) . Preliminary qualification 
tests are designed toverify the performance of individual components prior 
to an integrated formal qualification of the complete computer program CEI. 
The PQT phase is conducted incrementally by components in the same manner 
as the Critical Design Review. Figure 3 depicts the relationship between 
CDR and the Category I test program. The crosshatched blocks in Figure 3 
indicate coding of individual computer program components. The 
Preliminary Qualification Tests are modular and a “building block 11 
effect occurs as testing progresses. As each computer program component 
is added and each PQT conducted, increased confidence develops in the 
computer program CEI being tested. 

Formal Qualification Testing (FQT). Formal qualification tests 
represent the iinal step in Category Y testing of a computer program CEI. 
They insure that the complete CEI actually meets the Part I specification 
performance requirements. However, the necessity for formal qualification 
places more stringent requirements on the computer program’s"environment"; 
it must now leave the confines of an artificial world and enter the realm 
of the real world. 

Qualification testing of a complex computer program contract end 
item requires extensive use of simulation techniques. The use of these 
techniques is dictated by the high cost of providing overhead computer 
facilities or by the unavailability of new computers undergoing a parallel 
design and development effort. Although Preliminary Qualification Tests 
will make maximum use of simulation techniques, the Formal Qualification 
Tests will generally require live inputs, live outputs and operationally- 
configured equipment. A prerequisite, then, of FQT is usually the 
installation and checkout of the computer program CEI in an operationally- 
configured system at the Category II test site. To provide reliable 
data during FQT of a computer program CEI, fully installed and checked out 
equipment should be available. Subsequent to installation and checkout 
of the computer program CEI, FQT is conducted. The conclusion of FQT 
signals the end of the Category I test program. The computer program CEI 
should be fully qualified and all of the requirements of the Part I 
specification should be satisfied except for those requirements of the 
Part I specification that can only be demonstrated during a Category II 
system test. After successfully passing this phase of testing, the computer 
program is fully integrated into the system and is ready for system testing. 
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Figure 3 



























Category II (System) Testing , At the conclusion of Category I 
testing, the Air Force conducts an extensive Category II system test 
program with the objective of demonstrating that the system meets 
system performance/design requirements of the System Specification. 
Insofar as the computer programs are concerned, Category II testing 
will verify their compatibility with the system and their integrated 
performance in meeting system requirements in the live environment, 
with operational communications, personnel, etc. Residual and design 
errors discovered in this phase of testing are corrected by the 
contractor prior to the system becoming operational. 


SOME TYPICAL EXAMPLES 


An Expensive Education 

In the past the Air Force has had some painful experience in the 
acquisition of computer based systems. In the acquisition of one recent 
large computer based system, problems generated in the development of the 
information processing segment proved so great that they prevented the 
system from becoming operational. One of the factors that contributed to 
the failure of the project was the lack of a system specification. As a 
result no explicitly defined nor commonly understood set of system 
objectives was established. Consequently, specific system requirements 
could not be allocated to individual computer programs. The design docu¬ 
ments for the computer programs were generally not definitive enough and 
not subject to controls. The net effect was that a design requirements 
baseline was never established. The contractor was, to a certain degree, 
left to design and develop the computer programs on his own without 
detailed guidance and feedback from the Air Force resulting in computer 
programs that never did satisfy the users requirements even after repeated 
redesigns. The cost of the computer programs grew to five times the 
original estimates during the design and development. When the decision 
was made to delete the system from the inventory approximately $27,000,000 
had been spent on the system, of which about 55% represented computer 
program costs, and additional funds were needed to meet the system 
requirements. Even those elements of the system that operated properly 
suffered from inadequate and inaccurate documentation and thus could not 
be used to maximum effectiveness. It cannot be claimed that the use of 
a specific technique or group of techniques would have solved all of the 
problems associated with this system. It is clear, however, that the use 
of a systems approach that included the computer programs would have 
greatly increased the probability of achieving the design goals. 

Some Recent Improvements 

BUIC, an acronym for Back-Up Interceptor Control System, is an air 
defense system that enhances the survivability of North America*s air 
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defense in a post attack environment. The BUIC system has progressed in 
evolutionary steps from the manual BUIC I system to the sophisticated 
BUIC III system that incorporates a large modular digital computer. The 
BUIC system represents the first attempt at ESD to apply systems management 
to a total computer based system. 

Draft versions of the ESD Exhibit‘d on configuration management of 
computer programs were applied to the BUIC contracts. The system 
specification identified four computer program contract end items: 

Air Defense Computer Program CEI, Utility Computer Program CEI, System 
Exercise Computer Program CEI, and Confidence-Diagnostic Computer Program 
CEI. Specific system requirements were allocated to each of the computer 
program CEI*s and detailed design requirements with qualitative measures, 
where possible, were established for each CEI. Fault detection and 
isolation are typical examples of the requirements placed on the 
Confidence/Diagnostic CEI while requirements such as preparation of 
exercise tapes and control of system exercise missions were placed on the 
System Exercise CEI. In addition the interfaces between the various 
computer program and equipment CEI’s were defined in the individual 
design specifications. Particular emphasis was placed on the interface 
between the Air Defense and Confidence/Diagnostic computer programs for 
two reasons: first, the unique design of BUIC wherein two programs 
operate in parallel (Air Defense and Confidence/Diagnostic) using the 
redundant computer modules creates an exceptionally complex interfacej 
second, the Confidence/Diagnostic program was written by the computer 
manufacturer while the rest of the computer programs w r ere written by a 
second contractor. 

A Modular Computing System . It was during a BUIC II design review 
that the full system impact of computer programs was discovered. The 
equipment manufacturer had concluded that the system reliability 
requirements could not be met with the proposed equipments. Even the use 
of system redundancy would not provide the desired system Mean Time 
betv/een failure (MTBF) with the available equipment end items. Analysis 
of the modular equipment and the Confidence/Diagnostic computer programs 
led to a computer program-controlled modular computing system with 
redundant modules. The system, described by Blanton, is illustrated in 
figure [4 and shows the redundant modules. During normal operation the 
control of the computing modules is shared by two computer programs 
operating in parallel: an operational (Air Defense) program and a back-up 
(Confidence/Diagnostic) program. The operational program performs air 
defense functions, limited failure detection and system reconfiguration 
and startover. The back-up program exercises the modules in the back-up 
system and detects failures to maintain confidence in the redundant 
modules should they be needed in the "operational" system. To guard 
against failures in the "operational" system remaining undetected for 
long periods of time the modules are periodically switched between the 
"operational" and "back-up" systems. In BUIC the complete failure 


Ik 



RELIABILITY BLOCK DIAGRAM 
BUIC CENTRAL COMPUTING MODULES 



Figure 4 
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detection, failure isolation and system recovery is accomplished in 
less than 30 seconds. The 30 second recovery time does not seriously 
degrade system operation since the "operational" system is capable of 
restarting where it was halted without much loss of information. 
Blanton^ has calculated the MTBF of the modular system in figure h to 
be 87*858 hours. This represents a system 33 times more reliable than 
a duplexed system, yet the duplexed system requires a 23 % increase in 
modules. 

In this example the full value of computer programs within a system 
was realized early in the design stages, but only as a result of the 
equipment being unable to meet design requirements. 

Conclusion 


The broadening of systems management to include computer programs as 
well as equipment, facilities, etc. can provide a powerful tool for the 
technical manager. These techniques can provide definitive design 
requirements and increase technical control in the design and development 
of computer programs. They improve the computer programmer's position in 
the design process by insuring that his requirements, interfaces, etc. are 
properly emphasized from the system's conception through its attainment 
of operational status. 

In applying the systems approach one cannot equate computer programs 
with equipments; rather, the requirement for similar technical controls 
must be recognized with due consideration for the inherent differences 
between computer programs and eouipnent. To realize the full value of 
computer programs, increased emphasis must be placed on systems analysis 
of computer programs. As shown in the BUI.C system., the potential of 
computer programs is often far above that initially considered and 
achieving this potential may recuire much innovation. It is only 
through a systems approach that the full systems effectiveness of 
computer programs will be realized. 
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